Because of recent advances of control logic, image processing, and sound recognition technology, robots have broadened a scope of application from an industrial use to an entertainment use. Many entertainment robots such as a pet robot AIBO® produced by Sony Co. and a toy-humanoid robot i-SOBOT® produced by Tomy Company, Ltd. are released in the market. Among entertainment robots, a music-related robot (hereinafter referred to as music robot) is expected to become important in the future because music holds a prominent place as leisure and to grow in the future as well.
Among music robots, playing an instrument robot is a promising field where interactive entertainment is provided by playing in concert between humans and robots.
An expression method of the music robot is classified into (1) sound such as singing songs, (2) motion such as dancing, and (3) playing an instrument.
As a robot singing songs of the classification (1), a robot that counts beats is disclosed in non-patent literature 1 and a biped robot that sings while stepping to beats is disclosed in non-patent literature 2. The both robots use sound obtained from a microphone mounted on the robot as an input sound.
As a dancing robot of the classification (2), a biped robot that dances by imitating a whole human body motion is disclosed in non-patent literature 3. A robot that social-dances by arms and wheel movement is disclosed in non-patent literature 4. A robot that keeps steps to sounds listened to by itself is disclosed in non-patent literature 5.
With respect to a robot playing an instrument of the classification (3), there are reports on a solo player and a concert player. As for the solo player, a whole body humanoid robot WABOT-2™ that plays the keyboard but has no walking function is disclosed in non-patent literature 6. A humanoid robot that plays the violin and has both arms and an upper body is disclosed in non-patent literature 7. Most of the robots except for the WABOT-2™ of the non-patent literature 6 play not melody but a single sound. However, in recent years, a humanoid robot WF-4RIV™ that plays the flute in non-patent literature 9 and a robot WAS-1™ in non-patent literature 10 that has only artificial lips and fingers and plays the saxophone have actually played a complex melody. Task of these robots is inputting MIDI-form scores and target pitches and faithfully following them. With respect to playing in concert with the human, a simple concert robot that plays in turns is disclosed in non-patent literature 11, and improvisational concert robots are disclosed in non-patent literature 13.
Theremin is most different from musical instruments subjected by conventional studies. Theremin can be played without physical contact between a robot and an instrument. Therefore, special hardware such as artificial lips required for playing the flute and the saxophone and a precision finger mechanism required for playing the keyboard is not necessary. Theremin can be played by many robots that satisfy requirement of having two arms. Consequently, a thereminist system is expected to be highly portable because they are implemented on many existing robots.
There are two problems with playing the theremin:    (A1) there exists no physical reference point as a basis during playing, unlike a keyboard of piano and a fret of guitar,    (A2) relation between pitch and volume of theremin and position of both arms (hereinafter referred to as pitch characteristics and volume characteristics) changes depending on surrounding environments such as temperature and number of people (hereinafter referred to as environment capacitance). In other words, an adaptive control is required. This is because the pitch characteristics and the volume characteristics change in response to the environment capacitance, and the environment changes every second even if sufficient preparation is possible in advance. Although elaboration is possible if information about the environment is known in advance, a playing method adaptive to an unknown environment by a small number of measurement is indispensable for actually playing in various environments.
Especially for playing in concert, an adaptive playing method is a key to the concert because other players affect an environmental capacitance.
Ordinarily, the thereminist robot has to play melodies in such a state that it is in proximity to partners during the concert. Two requirements for realizing that are as follows:
(1) quick arm control, and
(2) adaptation to different environmental capacitance.
Item (1) is required for playing the melody. Because sounds of theremin continuously change, an audience cannot recognize sound lines played by a robot, unless the robot quickly moves its arm to a target position and stands still at a move destination. Item (2) is required for playing in concert where other players are nearby.
In consideration of these requirements, a quantitative pitch control has been conventionally studied. Approaches to pitch control are divided into two controls: feedback control shown in non-patent literature 14 and others and feedforward control shown in non-patent literature 15 and others.
The feedback control allows accurate pitch control because an arm position is adjusted by listening to theremin sound during a performance. However, the feedback control does not satisfy the above-mentioned requirements (1) and (2) due to the following reasons:
Firstly, it is impossible to play the melody in which target values continuously change in less than one second in some pieces of music, because it takes much time to reach a target pitch.
Secondly, it is difficult for an audience to recognize the melody being played because the pitch has to be gradually changed in sequence to realize the feedback control.
On the contrary, the feedforward control satisfies the requirement (1) because quick arm control is possible. However, it is indispensable to deal with robustness of a model used for estimating a target position of arm (the requirement (2)). For the feedforward control, an appropriate target position should be estimated without listening to an actual sound. The estimation is missed unless the changing environmental capacitance is embedded in a model.
A feedforward control method based on a lookup table is proposed in non-patent literature 15. In this method, a table for outputting an appropriate joint angle to all pitch names (e.g. C3, D4) being played is prepared in advance, and an arm is controlled based on the table during performance.